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ABSTRACT 



A content analysis instrument was developed to help 
educators select textbooks that deal with the Middle East and reflect the 
goals and standards published by UNESCO and the National Councils for Social 
Studies, Geography, and World History. Fifty- five goals and standards were 
identified from those published by the four organizations. Sixty 
college-level faculty and specialists in Middle East history rank ordered the 
55 goals and standards in terms of how important they thought the item was 
with respect to inclusion in college-level textbooks that deal with the 
Middle East. The 60 rankings were analyzed, with the mean and standard 
deviation calculated for each item and each category. Items were then 
rank- ordered from high mean to low within each category and from low standard 
deviation to high within each category. These rankings were used to generate 
a second instrument, which was sent to 184 participants. Results show 
agreement between the four organizations and specialists who teach about the 
Middle East with respect to inclusion in college- level textbooks that deal 
with the Middle East. (Contains 2 tables and 15 references.) (Author/SLD) 
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ABSTRACT 




This proposal focused on developing a content analysis instruments which would 
help educators select textbooks that dealt with the Middle East and which, at the 
same time, reflected the goals and standards published by UNESCO and the 
National Councils for Social Studies, Geography, and World History. 

The researcher first examined the sets of goals and standards published by the 
four organizations to see which goals or standards were essentially the same. 
Fifty-five goals and standards were identified. Then sixty college-level faculty 
and specialists in Middle East history rank-ordered the 55 goals and standards in 
terms of how important they thought the item was with respect to inclusion in 
college-level textbooks that deal with the Middle East. 

The researcher analyzed the 60 instruments that were returned, and calculated the 
mean and standard deviation for each item and each category. The items were 
then rank-ordered from high mean to low within each category and from low 
standard deviation to high within each category. These rankings were then used to 
generate a second instrument. The second instrument was sent out to the 
participants. The researcher found out that there was agreement among 
organizations such as UNESCO and United States National Councils for Social 
Studies, Geography, and World History, and specialists who teach about the 
Middle East with respect to inclusion in college-level textbooks that deal with the 
Middle East. 
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THE DEVELOPMENT OF A CONTENT ANALYSIS INSTRUMENT FOR 

ANALYZING COLLEGE-LEVEL TEXTBOOKS USED IN THE 
UNITED STATES TO TEACH ABOUT THE MIDDLE EAST 

Introduction 

There seems to be a clear need for students to learn about different peoples 
and cultures, and particularly those of the Middle East. One reason for this is that 
the Middle East is an area of the world that has been, and continues to be 
politically, an international hot spot. Since the United States is involved in the 
Middle East, it is beneficial for students in the U.S. to learn about the people and 
the cultures of the area. 

Even if the Middle East was not politically a hot spot, it would still be 
important for student to learn about its peoples and cultures because there has 
been a dramatic growth in international business and in the exchange of ideas 
worldwide. This growth has increased the need for people to learn more about the 
history and cultures of people and lands other than their own. Peaceful 
coexistence depends, to a great degree, on the extent to which people understand 
other peoples and their cultures. 

Many colleges, universities and other educational institutions in the United 
States are increasing their efforts to teach both preservice and inservice teachers 
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about other cultures. For example, in a study done by Michigan State Board of 
Education (1984), researchers explained that “one would not expect to find 
extensive coverage of other world areas per se in a textbook of American history. 
However, it is important for contemporary scholars and authors of textbooks to 
present American history within its international context” (p. 44). Given the 
increased interest in teaching about other cultures, there is likely to be increased 
interest in textbooks that deal with other cultures. This is so because textbooks are 
of major importance in providing information to students. This is substantiated by 
research studies that examined the amount of time students and teachers spend on 
textbook related activities. Wade (1993) indicates that, "a number of studies have 
found that students engage in textbook-related activities 70 to 95 percent of the 
time that they spend in classrooms" (p. 232). Siler (1987) explained that “teachers 
organize their daily teaching activities around the basic textbook. Curriculum 
planning and daily lesson planning tend to be determined by the basic textbook” 
(P- 78). 



Recent Statements Efforts 

In developing a content analysis instrument, the researcher wanted to 
include goals and standards that were as broadly representative as possible. 
While there are still no educational goals and standards that are universal, it 
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appears to this researcher that educators may be closer to agreement today 
than at any time. 

In recent years educators in organizations such as the United States 
National Councils for Social Studies, Geography, and World History 
(hereafter referred to as the Councils) and the United Nations Educational, 
Scientific and Cultural Organization (UNESCO) have developed sets of 
goals and standards that reflect what they believe students should know and 
be able to do. These goals and standards were used as a foundation for this 
content analysis instrument. 

These goals and standards are important for students to know because 
of many reasons. First, these standards and goals were selected and collected 
from many countries. They represent the world’s communities’ best 
thinking about what constitutes appropriate and challenging content. 

Second, these standards represent a consensus on what students should 
know and be able to do. For example the Geography standards developed a 
consensus process and model to develop its standards. This model involved 
“The consumers of education and the providers of education, including all 
of the major geography organizations in the United States” (p. 246). 

Third, these goals and standards represent the concerns and interests of 
both the public and private sectors, and government agencies. They are also based 
on the educational reform movement of 1980s, the subsequent articulation of the 
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National Governors Association’s goals in 1990 and the congressional passage of 
Goals 2000 in 1992. 

Fourth, these goals and standards that were developed by UNESCO and 
the National Councils are uniquely common in terms of their emphasis. The main 
focus of these organizations and agencies is on humanity. This commonality is 
evident in all the standards and objectives of these organizations. It is clearly 
demonstrated in the National Standards for World History: Exploring Paths to the 
Present . The developers of the world history standards explain that “these 
standards rest on the premise that our schools must teach a comprehensive history 
in which all standards may share. That means a history that encompasses 
humanity” (P. 4). 

Guidelines Developed by United Nations Educational, Scientific and Cultural 

Organization (UNESCO) 

Since 1947, (UNESCO) has been involved in setting up educational 
guidelines and objectives. These objectives and guidelines address different 
aspects of education. In 1974, at the 18 th General Conference session, which was 
held in Paris, UNESCO charted the following educational guidelines and 
objectives: 

Education should be infused with the aims and purposes set forth in the 



Charter of the United Nations, the Constitution of UNESCO and the 
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Universal Declaration of Human Rights, particularly Article 26, paragraph 2, 
of the last-named, which states 'Education shall be directed to the full 
development of the human personality and to the strengthening of respect for 
human rights and fundamental freedoms. It shall promote understanding, 
tolerance and friendship among all nations, racial or religious groups, and 
shall further the activities of the United Nations for the maintenance of 
peace', (pp. 3-4) 

There is clearly an increased interest in helping students learn about other 
cultures. What is not yet clear is the extent to which college-level textbooks used 
in the United States to teach about the Middle East, reflect the skills and 
understandings recommended by organizations, such as: UNESCO and the 
Councils. 



History of Content Analysis Methodology 
Krippendorff (1980) defines content analysis as a "research technique for 
making replicable and valid inferences from data to their context. As a research 
technique, content analysis involves specified procedures for processing scientific 
data. Like all research techniques, its purpose is to provide knowledge, new 
insights, a representation of facts and a practical guide to action" (p. 21). This 
technique was used as a guide to develop a content analysis instrument to 
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examine the extent to which textbooks that teach about the Middle East reflect 
objectives and standards developed by UNESCO and the Councils. 

Krippendorff (1980) discussed different components that constitute 
content analysis research designs. These components are: (a) Data making, (b) 
Unitization, (c) Sampling, (d) Recording, (e) Data reduction, (f) Inference, (g) 
Analysis, (h) Direct validation, (i) Testing for correspondence with other 
methods, and (j) Testing hypotheses regarding other data. (p. 52) 

Since 1938, many research studies have used content analysis as a method 
of investigation. These studies revealed many problems with history textbooks, 
such as: bias, discrimination, stereotypes, distortion, and omission. While these 
studies contributed immensely to the improvement of history textbooks (Michigan 
State Board of Education, 1984), they did not enhance the methods of analyzing 
textbooks. In 1949, Siler (1987) reported a study by the American Council on 
Education Intergroup Relations in Teaching Materials which “concluded that 
textbooks used throughout the United States were distressingly inadequate, 
inappropriate, and even damaging to intergroup relations” (p. 82). The American 
Council on Education study reported that “the report reviewed 315 textbooks 
ranging from grade four to grade fourteen, encompassing social studies texts as 
well as texts from several other disciplines.... The report presented no 
quantitative data from which to develop any measurable analysis” (p. 82). 
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During the 1 960s, content analysis studies became more specialized. 
Minority groups and organizations that were dissatisfied with textbooks 
conducted the bulk of content analysis studies. The topics of investigation 
included three basic parts: themes, groups, and historical events (Siler, 1987). 
While these studies have significantly contributed to raising the concerns of 
different minority groups, they were rudimentary in their analysis procedures. 

Siler (1987) evaluated different studies and found out that: “The literature 
encompassing the textbook treatment of groups, themes, and historical events 
indicated little use of content analysis as a process for research” (p. 95). 

Wade (1993) supported the findings of many studies, such as; Siler 
(1987), Garcia and Tanner (1985), and Wallen and Fraenkel (1988). Wade (1993) 
explained that: 

The problems with sampling, defining terms, analyzing data, and reporting 
results that Siler (1987) notes are clearly evident in this study. It also 
appears that some of the problems Wallen and Fraenkel (1988) observed 
in other types of social studies research, notably inappropriate 
methodology, sampling bias, and unfounded conclusions, are evident in 
content analysis research as well. (p. 247) 

More important, the literature points out the need for developing a content 
analysis instrument that is valid and reliable. In an analysis of research studies in 
content analysis, Wade (1993) stated that, "the overall quality of content analysis 
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research during the 10-year period under study is decidedly disappointing" (p 
247). According to Wade’s study, content analysis suffers from "notably 
inappropriate methodology, sampling bias, and unfounded conclusions" (p. 247). 
Wade stressed that in content analysis methodology, "it is imperative that a 
systematic approach to content analysis be employed" in order to reduce the 
researcher's subjectivity, (p. 248) 

General Content Problems in Textbooks that Deal with the Middle East 

The bulk of content analysis studies that dealt with textbooks on the 
Middle East were concerned with one specific issue-bias. Others have looked at 
the issue of omission. In general, researchers have used content analysis to 
analyze structural and organizational patterns of information, content coverage, 
and instructional procedures. The articles evaluated for this study reflect those 
wide interests and objectives. The following is a summary and an analysis of 
those studies. 

A widely researched issue in the content analysis literature dealt with how 
people of Middle East are portrayed in American history textbooks. Researchers 
examined both elementary and secondary history textbooks for issues, such as: 
stereotypes, biases, image distortion, discrimination and omissions. Most of these 
studies have substantially supported the hypotheses that American history 
textbooks present distorted images, biases, and stereotypes about the people of the 
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Middle East. Dhand (1988), for example, stated that “some American teachers are 
biased in their views of the peoples and cultures of the Middle East, and 
secondary school textbooks published a decade or two ago handed down ancient 
prejudices and even originated some of their own” (p. 26). 

The Michigan Board of Education study (1984) examined ten eighth-grade 
U.S. history textbooks to determine the extent to which they reflect the pluralistic 
and diverse nature of the United States. Also, it looked at the potential of those 
textbooks to promote global interdependence and if such textbooks were 
educationally sound. This study concluded that “the international aspects of our 
history and of our contemporary society are grossly underrepresented. Reference 
to other peoples and nations of the world are weak and frequently present 
distorted views of those cultures” (Michigan State Board of Education, 1984, p. 
44). 

Many of the studies that were examined called for investigating textbooks 
that teach about the Middle East in terms of the content presentation and content 
coverage of culture, geography, economics, environment, peace, and minorities. 

Menconi (1981) examined the statements of misconception about the Arab 
World in textbooks. The study concluded that social studies teachers held a wide 
range of misconceptions about the Arab world. 

Menconi's study is important in pointing out the level of misunderstanding 
that teachers have about the Arab World; however, it falls short in two respects. 
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First, it did not analyze the textbooks' content coverage. Second, it dealt with only 
one cultural group-the Arabs. Other Middle Eastern cultures, such as: Turks, 
Iranians and Jews, were not included in Menconi’s study. The focus on a specific 
group does not give us a clear idea how the whole region is treated in terms of 
misconceptions about various cultures. 

Nabti's (1981) study dealt with examining the coverage of the Arab World 
in American secondary school world studies textbooks. He focused on a specific 
set of questions that dealt with understanding culture, history, religion, and 
politics. This study provided a more current, comprehensive and systematic 
evaluation of textbook coverage of the Arab World than was previously done. 
However, this study, which reduced the Middle East representation to a specific 
group of people, was limited to the secondary school level. 

In another study, Suleiman (1977) examined American images of Middle 
East peoples. He has identified the following important feet about textbooks' 
coverage of high school World History courses: 

World History textbooks usually devote 8-10% of their coverage to the 
history, civilization and politics of the Middle East. Often, most of the 
material deals with ancient civilizations. However, teacher interest and 
coverage in the classroom tend to center on the more recent, 20 th century 
period. (Suleiman, 1 977, p. 55) 
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The importance of such conclusions stems from the fact that they point out the 
need for investigating the coverage of preservice teachers' textbooks at 
universities and colleges. Such knowledge about textbook coverage is important 
because the level of teachers' understanding seems to be directly related to, and 
reflective of, textbook content. While these studies are significant in addressing 
specific issues, such as: bias, representation, and interdependence, they do not 
deal well with a major concern, namely, to what extent does the textbook content 
reflect content that is valid. In order to examine the content of college-level 
textbooks that teach about the Middle East in the United States, this study aimed 
at developing a content analysis instrument that is based on goals and standards 
published by UNESCO and the Councils. 

Purpose of the Study 

The purposes of this study were threefold. The first purpose was to 
examine the sets of goals and standards published by UNESCO and the Councils, 
and determine which, if any, of the goals and standards are common to all four 
sets. The second purpose was to have college-level faculty and specialists in 
Middle East history rank-order those common goals and standards in terms of 
their perceived importance for inclusion in college-level textbooks that deal with 
the Middle East. The third and last purpose was to use that rank-ordered list to 
develop a content analysis instrument that will help educators identify college- 
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level textbooks concerning Middle East history and culture that reflect those goals 
and standards. 



Instrumentation and Method of Validation 
Preliminary Instrument 

The first step in this development was to examine the sets of goals and 
standards recommended by UNSECO and the Councils to see which, if any, goals 
or standards were essentially the same. Among the four sets, there were720 goals 
and standards. The researcher examined 200 standards and objectives 
recommended by UNESCO, 210 by the Council for Geography, 230 by the 
Council for World History, and 80 by the Council for Social Studies. Based on 
this examination the researcher found 55 goals and standards that were essentially 
the same. 

After analyzing the content of the list of 55 goals and standards, the 
researcher created four categories: (a) History; (b) Geography and Environment; 
(c) Culture; and (d) Peace, Ethnicity and Multiculturalism. Each goal or standard 
was then placed into an appropriate category. This resulted in 25 items in History, 
8 in Geography and Environment, 12 in Culture, and 10 in Peace, Ethnicity, and 
Multiculturalism. 

The researcher asked 1 84 participants to rate each of the items. The rating 
scale had five categories: (1) Not Important, (2) Not Very Important, (3) 
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Somewhat Important, (4) Important, and (5) Very Important. There also was a 
space for each participant to write in any objectives, skills or understandings that 
he or she believed to be crucial, but which were not on the form. This preliminary 
instrument was sent to the 1 84 people making up the sample group. 

Second Instrument 

The researcher wanted to construct a content analysis instrument that was 
not only valid and reliable, but that was also relatively short and easy to use. This 
meant that the final instrument would have to contain only those items that had 
the highest degree of consensus. To begin the analysis process, the researcher 
calculated the means and standard deviations of the items and categories. 

The researcher analyzed the 60 instruments that were returned, and 
calculated the mean and standard deviation for each item and each category. The 
items were then rank-ordered from high mean to low mean within each category 
and from low standard deviation to high standard deviation within each category. 
The same ranking was performed on the categories as wholes. The item means 
reflected the perceived value attached to each of the items and the item standard 
deviation reflected the degree of consensus of the respondents. 

The second instrument consisted of 50 items, 21 items in History, 8 in 
Geography and Environment, 1 2 in Culture, and 9 in Peace, Ethnicity, and 
Multiculturalism. 
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The second instrument was sent to the same group of participants and they 
were asked again rate each item. The researcher then calculated the means and 
standard deviations for the 60 surveys that were returned. Each item and each 
category was ranked-ordered from high mean to low mean and from low standard 
deviation to high standard deviation. 

The results of the second instrument were used to construct the final 
instrument. After constructing the final instrument, the researcher used the test- 
retest technique to calculate the reliability coefficient. In addition, the percentages 
of rater agreement were calculated for the final selected items. In order to 
calculate the percentages of rater agreement, two methods were used. First, items 
that were rated the same on the first and second instruments were calculated. 
Second, items that were rated with one point difference between the first and the 
second administration of the instrument were calculated. 

Reliability of the Instrument 

The reliability of the final instrument was tested by using two methods: 
test-retest and raters' agreement techniques. 

In order to calculate the percentages of rater agreement, two methods were 
used. First, the researcher matched items on instruments one and two that had the 
same ratings. Then, the researcher matched items that differed by no more than 
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one point on instruments one and two. In addition the overall correlation 
coefficient was calculated. 

This test-retest technique was used because, according to Weisberg, et al. 
(1989), "the best way to assess the reliability of a question is by comparing 
answers people give to it on one occasion with the answers those same people 
give to it a short time later" (p. 76). DeVellis (1991) argued that, "another two- 
score method of computing reliability involves the temporal stability of a 
measure, or how constant scores remain from one occasion to another" (p. 37). 
The Test-retest reliability coefficient was used to calculate the ranking of the 
items that were common to the first and second instruments. Thus, the 
participants' responses on the first and second instrument were compared to 
determine their consistency. 

Validity of the Instrument 

The validity of this instrument is based on the following points. First, 
the items making up the preliminary instrument were drawn from the sets of 
goals and standards recommended by UNESCO and the Councils for Social 
Studies, Geography, and World History. These, presumably, represent the 
best thinking about what constitutes appropriate and challenging content by 
a large number of scholars specializing in Middle East history. These 
standards and goals were selected and collected from many countries. 
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Methods and Results 
Development of Instrument One 

To ensure that the content analysis instrument being developed was not 
subjective, the researcher turned to the sets of goals and standards developed by 
UNESCO and the United States National Councils for Social Studies, Geography, 
and World History. These goals and standards reflected what large numbers of 
subject area specialists believed students needed to know about other peoples, 
cultures, and places. 

Among the four sets, there were720 goals and standards. The researcher 
examined each of these to see if any of them were the same or nearly so across all 
four sets and were, at the same time, appropriate for use in a content analysis 
instrument. Fifty-five items were found and these were arranged into four 
categories: (1) History - 25 items; (2) Geography and Environment - 8 items; (3) 
Culture - 12 items; and (4) Peace, Ethnicity and Multiculturalism - 10 items. The 
researcher then developed a five category rating scale: (1) Not Important, (2) Not 
Very Important, (3) Somewhat Important, (4) Important, and (5) Very Important. 

Analysis of Instrument One 

In May of 1998, the researcher mailed copies of the instrument to 184 
participants. Each was asked to rate each goal and standard in terms of how 
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important they believed it was with respect to inclusion in textbooks that teach 
about the Middle East. Sixty forms were returned. 

The low number of returned surveys concerned the researcher. That low 
number may have been caused, at least in part, by the timing of the mailing. Since 
the instrument was mailed in the summer, many faculty members at colleges and 
universities were not available to respond. 

After receiving the sixty forms, the researcher computed the means and 
standard deviations of each of the 55 items and of the categories as wholes. Items 
with high means and low standard deviations were considered to be items that 
respondents agreed were important and should be reflected in the content of texts 
dealing with the Middle East. Participants were given the opportunity to add 
objectives and standards that they felt were important but which were not 
included in the instrument, but none were added. The researcher interprets this to 
mean that the items on the content analysis instrument seem to adequately reflect 
what the raters believed that students should know about the Middle East. 

Development of Instrument Two 

After analyzing the first instrument the researcher established a mean 
cutoff point of 3.50 and a standard deviation cutoff point of 1.30. Items that met 
or exceeded these cutoff points were those that made up the second instrument. 
The use of these more rigorous cutoff points resulted in the elimination of five 
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items. The reduced number of items helped to ensure that the second instrument 
dealt only with those items that most raters believed were most important. The 
shorter instrument also helped in arriving at a final instrument that would be not 
only valid and reliable, but also relatively short and easy to use. 

All 60 participants completed and returned the second instrument. The 
analysis of the second instrument returns showed that most of the item means 
were higher than the corresponding means on the first instrument and that most of 
the standard deviations were less than the corresponding standard deviations on 
the first instrument. The analysis also showed that the mean of 4.34 for the 
category of Peace, Ethnicity and Multiculturalism was the highest of the category 
means. 

Development of the Final Instrument 

After analyzing the second instrument, the researcher established new and 
more rigorous cutoff points as criteria for including items on the final instrument. 
The cutoff point for the mean was raised from 3.50 to 3.90 and the cutoff point for 
the standard deviation was lowered from 1.30 to 1.12. The use of these more 
rigorous cutoff points resulted in the elimination of 25 items. 

The 25 items on the final instrument were those that received means of 
3.90 and higher and standard deviations of 1.12 and lower. The analysis also 
showed that the mean for the category of Peace, Ethnicity and Multiculturalism 
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was the highest, 4.34. The mean for the category of Culture was 4.10. The 
Geography and Environment category had a mean of 4.04, followed by history 
with a mean of 4.01 (See table 2). 

Calculation of Reliability 

The reliability of the items on the final instrument was calculated in three 
ways. First, the researcher examined the extent to which raters rated each item the 
same on instruments one and two. This analysis showed that while all the items 
were rated the same by some raters, none were rated the same by all the raters. 

The extent of perfect agreement ranged from 31 raters (51%) for item one, to 15 
raters (25 %) for items eight and 16. 

The item analysis showed that in every case where there was a rating 
change, the rating went up one point. Consequently, the researcher did a second 
analysis in which those items for which there was perfect agreement were 
combined with those items for which the ratings differed by no more than one 
point. When analyzed in this way, the agreement ranged from 49 raters (81%) for 
items 14 and 15, to 35 (58%) for item seven. The overall percentage of agreement 
using the combined ratings was 73.48. 
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Test-retest Reliability 

A third method used to calculate the overall reliability of the final 
instrument was the Test-retest method. The researcher reviewed instruments one 



and two and extracted the 25 items that made up the final instrument. The test- 
retest method was then used to assess the reliability of the ratings of those items. 
The researcher found that there was considerable variability among the 



item-by-item reliability coefficients (See Table 1). The interpretation of this 
inconsistency is based on Hinkle et al (1994) rule of thumb for interpreting the 
size of a correlation coefficient. Hinkle et al (1994) provided the following chart 



for interpreting the size of a correlation coefficient: 



Size of Correlation 
.90 to 1.00 (-.90 to -1.00) 
.70 to .90 (-.70 to -.90) 
.50 to .70 (-.50 to -.70) 
.30 to .50 (-.30 to -.50) 
.00 to .30 (.00 to -.30) 



Interpretation 

Very high positive (negative) correlation 
High positive (negative) correlation 
Moderate positive (negative) correlation 
Low positive (negative) correlation 
Little if any correlation, (p 1 19) 



The reliability correlation coefficients for the 25-items ranged between 



.342 and -.168. The overall reliability coefficient was .40 (See Table 1). The 
researcher found a number of reasons that might explain these unexpectedly low 



correlation coefficients. For example, DeVellis (1991) explained that a problem 



with Test-retest reliability: 

is that what happens to the scores over time may or may not have to do with 
the error-proneness of the measurement procedure.... For example, if a 
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purported anxiety measure was influenced by social desirability as well as 
anxiety, scorers might remain constant despite variation in anxiety. The 
stability in scores, reflected in high correlation across occasions of 
administration, would not be the result of invariance in the phenomenon 
interest. Alternatively, the phenomenon may not change while scores on the 
measure do; that is, the scale could be unreliable.... Kelly and McGrath 
(1988) have identified four factors that are confounded when one examines 
two sets of scores on the same measure, separated in time. These are (a) real 
change in construct of interest (e.g., a net increase in average level of anxiety 
among a sample of individuals), (b) systematic oscillation in the phenomenon 
(e.g. variations in anxiety, around some constant mean, as a function of time 
of day), (c) changes attributable to differences in subjects or measurement 
methods rather than the phenomenon of interest (e.g., fatigue effects that 
cause items to be misread), and (d) temporal instability due to the inherent 
unreliability of the measurement procedure. Only (d) is unreliability, (p. 38) 
Another possible factor that might have led to the low reliability 
coefficient is that the test-retest reliability technique is based on the principle that 
there is one right answer to each question and that if instruction was successful, 
many, if not all, students will answer each question correctly. However, in this 
study, there were no right or wrong answers. The participants' responses to items 
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on the first and second analysis instruments reflected opinions, not right or wrong 
answers. 

The analysis of the 25-item final instrument showed that the mean ratings 
increased between the two administrations (See Table 2). The increase in the 
means showed that when participants rated the items a second time, their 
perception of the importance of the items tended to increase. While this decreased 
the final reliability coefficient, it did show that the items on the final instrument 
were given high ratings and that those ratings reflected a high degree of 
agreement among the raters. 



Conclusions 

As a result of engaging in this study and analyzing the data, the researcher 
reached the following conclusions: 

1 . There was no empirically based content analysis instrument available 
that college-level educators could use to select textbooks that dealt with 
the Middle East. 

2. One way to construct a content analysis instrument that would be as 
valid as possible, would be to have it reflect the goals and standards that 
UNESCO and the Nation Councils for Social Studies, Geography, and 
World History believed students should know and be able to do. 
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3. There were some items that were the same, or nearly so, across the sets 
goals and standards put forth by UNESCO and the Councils and that 
were appropriate for inclusion on a content analysis instrument. 

4. By subjecting to repeated analysis those items that were common to all 
four sets of goals and standards, it was possible to construct a content 
analysis instrument that reflected content that should be included in 
college-level textbooks dealing with the Middle East. 

Recommendations for Future Research 

1 . In this study, a content analysis instrument was developed to analyze 
textbooks that teach about the Middle East at colleges and universities in 
the United States. However, the study stopped short of field-testing the 
instrument by having educators use it to analyze actual textbooks. 
Therefore, it is recommended that a study be conducted to determine 
whether the content analysis instrument developed in this study can be 
used to help educators consistently select textbooks that reflect in the 
goals and standards on the content analysis instrument. 

2. When selecting participants for this study, the researcher did not take 
into consideration variable such as; race, religion, gender, or nationality. 
However, these variables may have influenced the ratings and, 
consequently, the test-retest reliability coefficient. Therefore, it is 
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recommended that a study be conducted in which a stratified sample is 
selected so that the effect of variables such as; race, religion, gender, and 



nationality can be determined. 
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Table 1 

Test-retest Correlation Coefficient of Final Instrument (N = 60) 



Item Correlation 

Coefficient 



A. Category - History ( 1 -5) 

1 . Explain how the mandate system altered patterns — .083 

2. Analyze how the World War I settlement — .143 

3 . Explain how international conditions affected .... .169 

4. Assess the role and general effects of ... . .058 

5. An introduction to the major problems of the .... .342 

B. Category - Geography and Environment (6-11) 

6. Locate and describe varying landforms and .... -.168 

7. Identity some of the reasons for changes in the ... . -.104 

8. Examine political structures and governments .... .047 

9. Identify how places take on symbolic meaning .... .174 

10. Explain how places and regions are stereotyped .259 

1 1 . Awareness of environmental problems and the .... -.076 

C. Category - Culture (12-16) 

1 2. Compare the development of religious and .... .035 

1 3 . Analyze the sources and development of Islamic .... .291 

1 4. Identify and describe selected historical .... .161 

1 5 . Develop critical sensitivities such as empathy .... .215 

1 6. Identify and interpret examples of stereotyping .... .072 

D. Category - Peace. Ethnicity and Multiculturalism 1 1 7-251 

17. Promote understanding, tolerance and friendship .... .303 

18. Promote international solidarity and ... . .283 

1 9. Understand and respect for all peoples .... -.051 

20. Awareness of the increasing global .... .106 

2 1 . Stress the inadmissibility of recourse to ... . .187 

22. Contribute to international understanding ... . .288 

23 . Familiarization with the processes by which .... -.114 

24. Analyze examples of conflict, cooperation .... -.165 

25. Analyze changes in the lives of women in ... . .110 
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Table 2 

Means and Standard Deviations of Final Instrument (N = 60) 



Item Mean SD 



A. Category - History ( 1 -5) Mean of Items = 4.01 

1. Explain how the mandate system altered patterns .... 4.03 .99 

2. Analyze how the World War I settlement — 3.93 .97 

3. Explain how international conditions affected .... 4.03 .97 

4. Assess the role and general effects of ... . 4.05 1 .09 

5. An introduction to the major problems of the .... 4.05 1.01 

B. Category - Geography and Environment (6-11) Mean of Items = 4.04 

6. Locate and describe varying landforms and .... 4.00 .93 

7. Identify some of the reasons for changes in the — 3.95 1.12 

8. Examine political structures and governments .... 4.18 .87 

9. Identify how places take on symbolic meaning .... 4.10 .96 

10. Explain how places and regions are stereotyped 4.10 .79 

1 1 . Awareness of environmental problems and the ... . 3.96 .95 

C. Category - Culture fl 2- 1 61 Mean of Items = 4,10 

12. Compare the development of religious and .... 3.91 .94 

13. Analyze the sources and development of Islamic .... 3.95 1.04 

14. Identify and describe selected historical .... 4.20 .89 

1 5 . Develop critical sensitivities such as empathy .... 4.2 1 .90 

16. Identify and interpret examples of stereotyping .... 4.25 .89 

D. Category - Peace, Ethnicity and Multiculture ( 1 7-251 Mean of Items = 4.34 

17. Promote understanding, tolerance and friendship ... . 4.16 .97 

18. Promote international solidarity and .... 4.13 .99 

1 9. Understand and respect for all peoples .... 4.35 .81 

20. Awareness of the increasing global .... 4.41 .72 

2 1 . Stress the inadmissibility of recourse to ... . 4.31 1 .08 

22. Contribute to international understanding .... 4.40 .94 

23. Familiarization with the processes by which .... 4.36 .84 

24. Analyze examples of conflict, cooperation .... 4.40 .84 

25. Analyze changes in the lives of women in . . .. 4.58 .64 
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FINAL INSTRUMENT/FULL TEXT 



HISTORY (1-5) 

1 . Explain how the mandate system altered patterns of European colonial rule in 
Africa and the Middle East, (world History Standards). 

2. Analyze how the World War I settlement contributed to the rise of both Pan- 
Arabism and nationalist struggles for independence in the Middle East, (world 

History). 

3. Explain how international conditions affected the creation of Israel, and 
analyze why persistent conflict developed between Israel and both Arab 
Palestinians and neighboring states, (world History standards). 

4. Assess the role and general effects of imperialism, colonization, and 
decolonization on the economic and political developments of the nineteenth 
and twentieth centuries (e.g., European disregard for existing Middle Eastern 
political boundaries in the organization of colonies and subsequent 

independent nations). (Geography Standards). 

5. An introduction to the major problems of the contemporary world: In order 
that learners have a better grasp of the political, social, cultural, economic, 
demographic, environmental and other issues of modem societies and to 
improve their ability to cope with them, (unesco). 

GEOGRAPHY AND ENVIRONMENT (6-11) 



6 . 

7. 

8 . 



9. 

10 

11 . 



Locate and describe varying landforms and geographic features, such as 
mountains, plateaus, islands, rain forests, deserts, and oceans, and explain 
their relationships within the ecosystem, (social studies standards). 

Identify some of the reasons for changes in the world's political boundaries. 

(Geography Standards). 

Examine political structures and governments as regional systems (e.g., the 
hierarchy of political units such as village, town, city, county, state, and 
country; the hierarchy of political party structure-precinct, ward, county, 

State, and national levels). (Geography Standards). 

Identify how places take on symbolic meaning (e.g., Jerusalem as a holy city 
for Muslims, Christians, and Jews). (Geography standards). 

Explain how places and regions are stereotyped. (Geography standards). 

Awareness of environmental problems and the accelerating damage to the 
natural and cultural environment, (unesco) 
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CULTURE (12-16) 

12. Compare the development of religious and ethical belief Systems in the 
civilizations of the Euphrates and the Nile Rivers and how they legitimized 
the political and social order, (world History Standards). 

13. Analyze the sources and development of Islamic law and the influence of 
Islamic law and Muslim practice on such areas as family life, moral behavior, 
marriage, women's status, inheritance, justice, and slavery, (world History 

Standards). 

14. Identify and describe selected historical periods and patterns of change 
within and across cultures, such as the rise of civilizations, the development 
of transportation systems, the growth and breakdown of colonial systems, 

and Others. (Social Studies Standards). 

15. Develop critical sensitivities such as empathy and skepticism regarding 
attitudes, values, and behaviors of people in different historical contexts. 

(Social Studies Standards). 

16. Identify and interpret examples of stereotyping, conformity, and altruism. 

(Social Studies Standards). 

PEACE. ETHNICITY. AND MULTICULTURALISM (17-25) 

1 7. Promote understanding, tolerance and friendship among all nations, racial or 
religious groups, (unesco). 

1 8. Promote international solidarity and co-operation, which are necessary in 
solving the world problems affecting the individual's and communities' life 
and exercise of fundamental rights and freedoms rights and freedoms. 

(UNESCO). 

19. Understand and respect for all peoples, their cultures, civilizations, values 
and ways of life, including domestic ethnic cultures and cultures of other 
nations, (unesco). 

20. Awareness of the increasing global interdependence between peoples and 
nations, (unesco). 

21 . Stress the inadmissibility of recourse to war for purposes of expansion, 
aggression and domination, or to the use of force and violence for purposes 
of repression, (unesco). 

22. Contribute to international understanding and strengthening of world peace 
and to the activities in the struggle against colonialism and neo-colonialism 
in all their forms and manifestations, and against all forms and varieties of 
racialism, fascism, and apartheid as well as other ideologies which breed 
national and racial hatred and which are contrary to the purposes of this 
recommendation, (unesco). 
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23. Familiarization with the processes by which cultures spread and evolve and 
the recognition of their equal dignity and the indissoluble link between 
cultural heritages and contemporary culture, (unesco). 

24. Analyze examples of conflict, cooperation, and interdependence among 

groups, Societies, and nations. (Social Studies Standards). 

25. Analyze changes in the lives of women in both industrialized and developing 
countries since World War II and assess the extent to which women have 
progressed toward social equality and economic opportunity, (world History 
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